Neural collapse under cross-entropy loss
نویسندگان
چکیده
We consider the variational problem of cross-entropy loss with n feature vectors on a unit hypersphere in Rd. prove that when d≥n−1, global minimum is given by simplex equiangular tight frame, which justifies neural collapse behavior. also that, as n→∞ fixed d, minimizing points will distribute uniformly and show connection frame potential Benedetto & Fickus.
منابع مشابه
ESTIMATING THE MEAN OF INVERSE GAUSSIAN DISTRIB WTION WITH KNOWN COEFFICIENT OF VARIATION UNDER ENTROPY LOSS
An estimation problem of the mean µ of an inverse Gaussian distribution IG(µ, C µ) with known coefficient of variation c is treated as a decision problem with entropy loss function. A class of Bayes estimators is constructed, and shown to include MRSE estimator as its closure. Two important members of this class can easily be computed using continued fractions
متن کاملEstimation of Lower Bounded Scale Parameter of Rescaled F-distribution under Entropy Loss Function
We consider the problem of estimating the scale parameter &beta of a rescaled F-distribution when &beta has a lower bounded constraint of the form &beta&gea, under the entropy loss function. An admissible minimax estimator of the scale parameter &beta, which is the pointwise limit of a sequence of Bayes estimators, is given. Also in the class of truncated linear estimators, the admissible estim...
متن کاملCOLLAPSE BEHAVIOUR OF PLATES UNDER THRUST
In line with the current research on the ultimate strength assessment of ship hull girders, careful understanding of the collapse behaviour of unstiffened plate elements within the ship hull girders is an important and necessary task. In this paper, the collapse behaviour of unstiffened plates under longitudinal and transverse thrust is studied by elastoplastic large deflection FEM and the char...
متن کاملInfrastructure topology optimization under competition through cross-entropy
In this article, we study a two-level non-cooperative game between providers acting on the same geographic area. Each provider has the opportunity to set up a network of stations so as to capture as many consumers as possible. Its deployment being costly, the provider has to optimize both the number of settled stations as well as their locations. In the first level each provider optimizes indep...
متن کاملCross Entropy Error Function in Neural Networks: Forecasting Gasoline Demand
This paper applies artificial neural networks to forecast gasoline consumption. The ANN is implemented using the cross entropy error function in the training stage. The cross entropy function is proven to accelerate the backpropagation algorithm and to provide good overall network performance with relatively short stagnation periods. To forecast gasoline consumption (GC), the ANN uses previous ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Applied and Computational Harmonic Analysis
سال: 2022
ISSN: ['1096-603X', '1063-5203']
DOI: https://doi.org/10.1016/j.acha.2021.12.011